Achieving Dextrous Grasping by Integrating Planning and Vision-Based Sensing

This paper deals with the automation of dextrous grasping in a partly known environment using a stereo vision system and a multiingered hand mounted on a robot arm. EEective grasping requires a combination of sensing and planning capabilities: sensing to construct a well-adapted model of the situation and to guide the execution of the task, and planning to determine an appropriate grasping strategy and to generate safe, feasible manipulator motions. We propose an integrated approach that combines computer vision, path planning, and manipulator control in three complementary activities: the reconstruction of task-oriented models of the workspace, the determination of appropriate grasping conngurations from computed`preshapes' of the hand, and the automatic generation and execution of hand/arm motions using a hybrid geometric path planner and a hybrid control system. This paper outlines the architecture of our system, discusses the new models and techniques we have developed, and nishes with a brief description of work-in-progress on the implementation and some preliminary experimental results.